Moka
Moka is a fast, concurrent cache library for Rust. Moka is inspired by Caffeine (Java) and Ristretto (Go).
Moka provides cache implementations that support full concurrency of retrievals and a high expected concurrency for updates. Moka also provides a not thread-safe cache implementation for single thread applications.
All caches perform a best-effort bounding of a hash map using an entry replacement algorithm to determine which entries to evict when the capacity is exceeded.
Features
- Thread-safe, highly concurrent in-memory cache implementations:
- Blocking caches that can be shared across OS threads.
- An asynchronous (futures aware) cache that can be accessed inside and outside of asynchronous contexts.
- A not thread-safe, in-memory cache implementation for single thread applications.
- Caches are bounded by the maximum number of entries.
- Maintains good hit rate by using an entry replacement algorithms inspired by
Caffeine:
- Admission to a cache is controlled by the Least Frequently Used (LFU) policy.
- Eviction from a cache is controlled by the Least Recently Used (LRU) policy.
- Supports expiration policies:
- Time to live
- Time to idle
Usage
Add this to your Cargo.toml
:
[]
= "0.3"
To use the asynchronous cache, enable a crate feature called "future".
[]
= { = "0.3", = ["future"] }
Example: Synchronous Cache
The thread-safe, blocking caches are defined in the sync
module.
Cache entries are manually added using insert
method, and are stored in the cache
until either evicted or manually invalidated.
Here's an example of reading and updating a cache by using multiple threads:
// Use the synchronous cache.
use Cache;
use thread;
Example: Asynchronous Cache
The asynchronous (futures aware) cache is defined in the future
module.
It works with asynchronous runtime such as Tokio,
async-std or actix-rt.
To use the asynchronous cache, enable a crate feature called "future".
Cache entries are manually added using an insert method, and are stored in the cache until either evicted or manually invalidated:
- Inside an async context (
async fn
orasync
block), useinsert
orinvalidate
method for updating the cache andawait
them. - Outside any async context, use
blocking_insert
orblocking_invalidate
methods. They will block for a short time under heavy updates.
Here is a similar program to the previous example, but using asynchronous cache with Tokio runtime:
// Cargo.toml
//
// [dependencies]
// moka = { version = "0.3", features = ["future"] }
// tokio = { version = "1", features = ["rt-multi-thread", "macros" ] }
// futures = "0.3"
// Use the asynchronous cache.
use Cache;
async
Avoiding to clone the value at get
For the concurrent caches (sync
and future
caches), the return type of get
method is Option<V>
instead of Option<&V>
, where V
is the value type. Every
time get
is called for an existing key, it creates a clone of the stored value V
and returns it. This is because the Cache
allows concurrent updates from threads so
a value stored in the cache can be dropped or replaced at any time by any other
thread. get
cannot return a reference &V
as it is impossible to guarantee the
value outlives the reference.
If you want to store values that will be expensive to clone, wrap them by
std::sync::Arc
before storing in a cache. Arc
is a thread-safe
reference-counted pointer and its clone()
method is cheap.
use Arc;
let key = ...
let large_value = vec!; // 2 MiB
// When insert, wrap the large_value by Arc.
cache.insert;
// get() will call Arc::clone() on the stored value, which is cheap.
cache.get;
Example: Expiration Policies
Moka supports the following expiration policies:
- Time to live: A cached entry will be expired after the specified duration past
from
insert
. - Time to idle: A cached entry will be expired after the specified duration past
from
get
orinsert
.
To set them, use the CacheBuilder
.
use CacheBuilder;
use Duration;
Hashing Algorithm
By default, a cache uses a hashing algorithm selected to provide resistance against HashDoS attacks.
The default hashing algorithm is the one used by std::collections::HashMap
, which
is currently SipHash 1-3, though this is subject to change at any point in the
future.
While its performance is very competitive for medium sized keys, other hashing algorithms will outperform it for small keys such as integers as well as large keys such as long strings. However those algorithms will typically not protect against attacks such as HashDoS.
The hashing algorithm can be replaced on a per-Cache
basis using the
build_with_hasher
method of the CacheBuilder
. Many alternative algorithms are
available on crates.io, such as the aHash crate.
Minimum Supported Rust Versions
This crate's minimum supported Rust versions (MSRV) are the followings:
Enabled Feature | MSRV |
---|---|
no feature (default) | Rust 1.45.2 |
future |
Rust 1.46.0 |
If no feature is enabled, MSRV will be updated conservatively. When using other
features, like future
, MSRV might be updated more frequently, up to the latest
stable. In both cases, increasing MSRV is not considered a semver-breaking
change.
Road Map
-
async
optimized caches. (v0.2.0
) - Cache statistics. (Hit rate, etc.)
- Upgrade TinyLFU to Window TinyLFU.
- The variable (per-entry) expiration, using a hierarchical timer wheel.
About the Name
Moka is named after the moka pot, a stove-top coffee maker that brews espresso-like coffee using boiling water pressurized by steam.
License
Moka is distributed under either of
- The MIT license
- The Apache License (Version 2.0)
at your option.
See LICENSE-MIT and LICENSE-APACHE for details.